Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make tests less flaky #4581

Merged
merged 1 commit into from
Jul 27, 2023
Merged

Make tests less flaky #4581

merged 1 commit into from
Jul 27, 2023

Conversation

janezpodhostnik
Copy link
Contributor

@janezpodhostnik janezpodhostnik commented Jul 24, 2023 •
edited
Loading

The problem is that EpochCommit event is of nondeterministic byte size. This is because of the changes to the random.

@janezpodhostnik janezpodhostnik self-assigned this Jul 24, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Jul 24, 2023 •
edited
Loading

FVM Benchstat comparison

This branch with compared with the base branch onflow:master commit 3f3da35

The command (for i in {1..7}; do go test ./fvm ./engine/execution/computation --bench . --tags relic -shuffle=on --benchmem --run ^$; done) was used.

Collapsed results for better readability

old.txtnew.txt
time/opdelta
ComputeBlock/16/cols/128/txes/1/max-concurrency-25.83s ± 0%5.68s ± 0%~(p=1.000 n=1+1)
ComputeBlock/16/cols/128/txes/2/max-concurrency-26.17s ± 0%6.02s ± 0%~(p=1.000 n=1+1)
 
us/txdelta
ComputeBlock/16/cols/128/txes/1/max-concurrency-22.85k ± 0%2.77k ± 0%~(p=1.000 n=1+1)
ComputeBlock/16/cols/128/txes/2/max-concurrency-23.01k ± 0%2.94k ± 0%~(p=1.000 n=1+1)
 
alloc/opdelta
ComputeBlock/16/cols/128/txes/1/max-concurrency-21.65GB ± 0%1.66GB ± 0%~(p=1.000 n=1+1)
ComputeBlock/16/cols/128/txes/2/max-concurrency-21.70GB ± 0%1.71GB ± 0%~(p=1.000 n=1+1)
 
allocs/opdelta
ComputeBlock/16/cols/128/txes/1/max-concurrency-222.0M ± 0%21.9M ± 0%~(p=1.000 n=1+1)
ComputeBlock/16/cols/128/txes/2/max-concurrency-222.6M ± 0%22.6M ± 0%~(p=1.000 n=1+1)
 

@janezpodhostnik janezpodhostnik force-pushed the janez/make-tests-less-flaky branch from 2c84bd7 to 490800e Compare July 25, 2023 19:29
@janezpodhostnik janezpodhostnik requested a review from tarakby July 26, 2023 13:30
@janezpodhostnik janezpodhostnik marked this pull request as ready for review July 26, 2023 13:30
@janezpodhostnik janezpodhostnik force-pushed the janez/make-tests-less-flaky branch from 490800e to 4f572a2 Compare July 27, 2023 13:13
@janezpodhostnik
Copy link
Contributor Author

bors merge

@bors
Copy link
Contributor

bors bot commented Jul 27, 2023

Build succeeded!

The publicly hosted instance of bors-ng is deprecated and will go away soon.

If you want to self-host your own instance, instructions are here.
For more help, visit the forum.

If you want to switch to GitHub's built-in merge queue, visit their help page.

@bors bors bot merged commit 86d1724 into master Jul 27, 2023
@bors bors bot deleted the janez/make-tests-less-flaky branch July 27, 2023 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tarakby tarakby tarakby approved these changes

@fxamacker fxamacker fxamacker approved these changes

@zhangchiqing zhangchiqing Awaiting requested review from zhangchiqing zhangchiqing is a code owner

@koko1123 koko1123 Awaiting requested review from koko1123

@ramtinms ramtinms Awaiting requested review from ramtinms

Assignees

@janezpodhostnik janezpodhostnik

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@janezpodhostnik @fxamacker @tarakby